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ON ^-REGULARIZATION IN LIGHT OF NASHED’S 
ILL-POSEDNESS CONCEPT 

JENS FLEMMING, BERND HOFMANN, AND IVAN VESELIC 


Abstract. Based on the powerful tool of variational inequalities, in recent 
papers convergence rates results on i 1 -regularization for ill-posed inverse prob¬ 
lems have been formulated in infinite dimensional spaces under the condition 
that the sparsity assumption slightly fails, but the solution is still in dr. In the 
present paper we improve those convergence rates results and apply them to 
the Cesaro operator equation in d? and to specific denoising problems. More¬ 
over, we formulate in this context relationships between Nashed’s types of 
ill-posedness and mapping properties like compactness and strict singularity. 


1. Introduction 

Variational sparsity regularization based on i 1 -norms became of significant in¬ 
terest in the past ten years with respect to inverse problems applications, e.g. in 
imaging (cf., e.g., [25]), but also with respect to the progress in regularization the¬ 
ory for the treatment of ill-posed operator equations in infinite dimensional Hilbert 
and Banach spaces (cf., e.g., [51171 H5l [TCI [251127] ). Moreover, with focus on spar¬ 
sity, the use of ^-regularization can be motivated for specific classes of well-posed 
problems, too (cf., e.g., JTO]). Based on the powerful tool of variational inequali¬ 
ties (also called variational source conditions), in [51 convergence rates results on 
^-regularization for linear ill-posed operator equations have been formulated in in¬ 
finite dimensional spaces under the condition that the sparsity assumption slightly 
fails, but the solution is still in i 1 . In the present paper, we improve those results 
and illustrate the improvement level with respect to the associated convergence 
rates for the Cesaro operator equation in ( 2 and for specific denoising problems. 
Since the variational inequality approach requires injectivity of the forward opera¬ 
tor (cf. [HI Proposition 5.6]), we restrict all considerations in this paper to injective 
linear forward operators which also ensure uniquely determined solutions for the 
corresponding linear operator equations. The focus on injectivity is also motivated 
by the fact that the ill-posedness concept of Section [2] suggested by M. Z. Nashed 
(cf. [25]) would require substantial technical refinements in a general Banach space 
setting if non-injective operators were included. 

Let A : X —¥ Y be an injective and bounded linear operator mapping between an 
infinite dimensional separable Hilbert space X and an infinite dimensional Banach 
space Y with norms || ■ ||x and || • ||y, respectively. We are searching for the uniquely 
determined solution x r £ X of the linear operator equation 

(1.1) Ax = y, x £ X, y £ 7 1(A), 
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where 1Z(A) denotes the range of A. Typically, instead of y only noisy data y s £ Y 
are available. In this context, we consider the deterministic noise model 

(1-2) \\y s -y\\Y<S 

with given noise level S > 0. 

If the operator A is nor mally solvable , i.e. its range is a closed subset in Y. then 
solving the equation 0 is a well-posed problem. Consequently, for injective A 
the inverse A~ l : TZ(A) Cf-fX exists and is also a bounded linear operator. If, 
on the other hand, the range of A is not closed, the inverse A~ x is an unbounded 
linear operator and solving the equation is an ill-posed problem. This means 
that small perturbations in the right-hand side may lead to arbitrarily large error 
in the solution. Then regularization methods are required for obtaining stable 
approximate solutions to equation & 

As usual we consider in the sequel the Banach spaces £ q , 1 < q < oo, and £°° of 
infinite sequences of real numbers with finite norms 

\ 1/9 

\x k \ q ) and ||£c|L°° := sup \x k \- 


F : = 


fee N 


The Banach space c 0 consists of the real sequences (xk)ke N with lim |ajfc | =0 and 

h—yoo 

is also equipped with the norm ||x|| Co := sup fcgN |xfc|. Moreover, by £° we denote 
the set of sparse sequences, where a sequence is sparse if only a finite number of 
components is not zero. For such sequences the number of nonzero components is 
given by 

'l, t> 0, 


OO 


x \\i° : =) J |sgn(x fe )| with sgn(f) := 


fc=l 


0 , 


t = 0, 
t < 0. 


Throughout this paper we fix an orthonormal basis in the Hilbert 

space X. By x = (xfe)fceN £ £ 2 we denote the infinite sequence of corresponding 
Fourier coefficients of x, i.e. 


: =E 


x k u 


(fc) 


fe=l 

The synthesis operator L : i 1 —> X defined as Lx := x is an injective bounded 
linear operator. Note that this operator can be extended to £ 2 , but in our setting 
we define it only on (}. 

The focus of our studies is on almost sparse solutions x' to equation (1.1). 
This means that only a finite number of coefficients x' k from the infinite sequence 
x t = (x|.)fcgN is relevant. Here we do not require strict sparsity, x t € £°, but we 
allow an infinite number of nonzero coefficients if they decay fast enough. Precisely, 
we assume ad £ l 1 throughout this paper. 

Introducing the operator A := A o L : l 1 —> Y our goal is to recover the solution 
ad £ i x of 


(1.3) 


Ax = y, x £ i 1 , y £ TZ(A), 


from noisy data y° £ Y satisfying (1.2). The following proposition shows that 
solving this equation is always an ill-posed problem, even if the original equation 
(1.1) is well-posed. 
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For the proof and for further reference we note that the synthesis operator L is 
a composition L = U o £ 2 of the embedding operator £ 2 '■ (- 1 —s > £ 2 and the Riesz 
isomorphism U : l 2 —> X. Thus, the operator A can be written as a composition 

(1.4) A = AoUo £ 2 

of three injective bounded linear operators. 


Proposition 1.1. The range of A is not closed. 

Proof. Assume that TZ(A) is closed. The full preimage of TZ(A) with r espe ct to 
A is 1Z{L). Thus, L has closed range, too. Looking at the composition (1.4), the 
full preimage of 7 Z(L) with respect to U is 1Z(S 2 ) = (}. Consequently, i L would 
be a closed subspace of £ 2 . Since £ l is dense in £ 2 this yields the contradiction 
t x =£ 2 . ' □ 


The proposition shows that equation (1.3) requires regularization in order to 
obtain stable approximate solutions. For this purpose we use a variant of variational 
regularization, called ^-regularization, where regularized solutions, denoted by x s a , 
are minimizers of the extremal problem 
1 


(1.5) 


P 


^Ax — y s \\y + ol IMI^i —> min, subject to x € l 1 . 


Here, 1 < p < 00 is some exponent and a > 0 is a regularization parameter. This 
regularization parameter is chosen in an appropriate manner, a priori as a = a(i5) 
depending on the noise level 5, or a posteriori as a = a(6,y s ) depending also on 
the present regularized solution y s (for details see Sections [5] and [ 3 ] below). 

We are interested in error estimates 

(1.6) \\x s a — a^l^i < C x ] y>((5) for all 0 < <5 < 6, 


where the positive constant C x t may depend on the solution but not on the noise 
level 6 > 0. The estimates (1.6) can be interpreted as convergence rates 

(1.7) \\x 5 a - = 0{tp{5)) as 5-^0 


with rate functions ip which are concave index functions. Following [Mj and [20] 
we call ip : (0, 00) —► (0, 00) an index function if it is a continuous and strictly 
increasing function with lim <p(t) = 0. 

t-H- 0 

The article is organized as follows: in the next section we briefly summarize 
results on existence, stability and convergence of ^-regularized solutions. Section[3] 
contains the main theorem of this paper which improves the result of Theorem 5.2 
from [9] and can lead to better convergence rates. Moreover, Section [4] provides 
some insight into the interplay between Nashed’s ill-posedness concept and mapping 
properties of the forward operator like compactness and strict singularity. In the 
final Section [5] we apply our findings to a problem of denoising type. 


2. Existence, stability and convergence of ^-regularized solutions 

From the general theory of Tikhonov regularization (cf., e.g., [19, Section 3], 
[291 Sections 4.1.1 and 4.1.2] and [221 Section 3.1]) one can infer the existence 
and stability of ^-regularized solutions x s a as well as its convergence for 6 —> 0 
to the uniquely determined solution ad of equation (1.3) for appropriate choices of 
the regularization parameter a > 0. For this purpose we summarize the results 
of Proposition 2.8 and Remark 2.9 from [9] in the following proposition taking 
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into account that the separable Banach space cq is a predual space of f. 1 , that the 
operator A : l 1 —> Y is sequentially weak*-to-weak continuous, and that l 1 satisfies 
the weak* Kadec-Klee property (for a proof see, e.g., [5] Lemma 2.2]). The weak*- 
to-weak continuity of A can be shown along the lines of the proof of Lemma 2.7 
in [9] when 1Z(A*) C cq is valid for the adjoint operator A* : Y* — > £°° to A. In 
our setting, 1Z(A*) C cq is a consequence of the weak convergence Au^ 0 in 

Y for the prescribed orthonormal system {u^}keN in the Hilbert space A', which 
implies the weak convergence of formula ( |3.11[ ) in Remark 3.6 below such that 
the corresponding part of the proof of B Proposition 2.4] applies even if ( |3.12| ) is 
violated. 

Proposition 2.1. For all 1 < p < oo, a > 0 and y s £ Y there exist uniquely 
determined minimizers x 5 a £ I 1 of the extremal problem [1.5). These l 1 -regularized 
solutions are always sparse, i.e. they satisfy 

xt € e°. 

Furthermore, they are always stable with respect to the data, i.e., small perturbations 
in y s in the norm topology of Y lead only to small changes in x s a with respect to 
the t 1 - norm. 

If tin ~t 0 and if the regularization parameters a n = a(S n ,y 6rl ) are chosen such 


that 




O-n ~> 0 

and 

then 



(2.1) 


lim 

n—> oo 


S p 


0 


lb 1 


3. Improved convergence rates 


The norm convergence (2.11 can be arbitrarily slow. In order to obtain con¬ 
vergence rates, also for the ^-regularization with regularized solutions x^ a defined 


as minimizers t o pr oblem (1.5), a link condition between the smoothness of the 
solution x' to (1.3) and the forward operator A is required. From the stud¬ 


ies and results of the recent paper [9] we immediately derive the following theo¬ 
rem, where this link condition is a range condition imposed on all unit sequences 
e( fe ) = (0,..., 0,1, 0,...), k £ N, with respect to the adjoint operator A*. We men¬ 
tion here that {eWjfcgN represents a Schauder basis in the Banach spaces l q for all 
1 < q < oo. 


Theorem 3.1. Let the operator A: i 1 —► Y from equation (1.3) be such that there 


exist elements f ^ £ Y*, k £ N, satisfying the range conditions 

(3.1) e (fe) =A*f {k \ 

Then a variational inequality 

(3.2) ||x - x t ||^i < ||:r||£i - ||x t ||*i + <pi{\\Ax - Ax^\\y) for all x £ I 1 
is valid for the concave index function 


= I4i + *£ii/ (fc) n- 

\fc=n+1 


k=1 


(3.3) 
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This yields the convergence rate 

(3.4) ||4 - x t ||^i = 0(ipi(S)) as 6 0 

for l 1 -regul arize d solutions x s a and for the uniquely determined solution x* € t' 1 
of equation (1.3) provided that the regularization parameter a = a(S,y s ) is chosen 
appropriately, e.g. according to the discrepancy principle 

(3-5) n6 <\\Ax s a{SyS) -y s \\Y <t 2 S 

for prescribed values 1 < T\ < T 2 < oo. 

Remark 3.2. For more details concerning the consequences of variational inequal¬ 
ities and the role of the choice of the regularization parameter for obtaining conver¬ 
gence rates in regularization we refer, for example, to [fjJj and 
Making use of Gelfand triples it was shown in [2] that, for a wide range of applied 
i nver se problems, the forward operators A are such that link conditions of the form 
(3.1) apply for all e^ k \ k £ N. On the other hand, the paper m gives counterex¬ 
amples where (3.1) fails for specific operators A, but alternative link conditions 
presented there can compensate this deficit. 

We improve the convergence rate obtained in the theorem above as follows: 


Theorem 3.3. Theorem 3.1 remains true if is replaced by tp 2 with 
(3.6) ip 2 (t) = 2 inf 

n£ N 


1 00 

J2 \x{\+t sup 

n 


l k=n+l 0fc fc e _t“b°d> 

k =1 

Y*J 


Proof. From [9j Lemma 5.1] we know that 
(3.7) ||2C - 01*11*1 - ||2c||*i + ||||^i < 2 

Observing 



\ xk - 


k =1 


\xk - 41 = (sgn(a:fc - x\)){x k - 4) 

= (sgn(x fe — 4 )) (s ( ^ ) , x x^)^oo X £i 
= (sgn(x k - 4 ))(f {k \Ax - Ax^) y *xy 
the second sum on the right-hand side can be estimated above by 

n n 

i Xfc - 41 = XX sgn ( Xfc _ 4 )) (/ (fe) > Ax - ax^) y *xy 


k=l 


k =1 


= ( ^2(sgn(x k - xl))f( k \Ax - Ax * 


\k =1 


Y*xY 


< 


^(sgn(x fe -4))4 fc ) 

k =1 

< 11 Ax — Ax^ ||y 


\\Ax-Ax*\\ y 


Y* 


sup 

afcG{ —1,0,1} 

k—l,...,n 


^2a k f 


(fc) 


k=1 


Y * 


(3.8) 
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and taking the infimum over all n £ N in the resulting inequality yields the varia¬ 
tional inequality (3.2). □ 

To understand the difference between ipi and ip 2 it may be helpful to note that 
tpi = ip 2 if all are pairwise collinear. On the other hand, in the particu¬ 
lar case that Y* = Y = t 2 and /^ = e^ k \ we have Y^k=i ||/^||y* = n while 

su Pa fc6 {-i,o,i} ||ELi a fc/ (fc) |lv* = V 7 ™- 

k—1 ,... ,n 

In Example |3.7| and in Section [5] we will show that the improved index function 
ip 2 yields better convergence rates for some equations than the original function <p\. 
The convergence rate result obtained in |13l can be improved in a similar way. 


Example 3.4 (Holder rates). If 


(3.9) 


\x\\<K 1 n ^ and 


k=n -\-1 


sup 

a fc e{—1,0,1} 

k=l,...,n 




k =1 


< K 2 n v 


for K\, K 2 > 0 and p, v > 0, then the convergence rate obtained in Theorem 3.3 
(3.10) Ika ~ = 0(6*+"') as <5 —>■ 0 

(cf. [3 Example 5.3]). 

Example 3.5 (exponential decay of solution components). If 


is 


\x\\ < K\ exp(—u 7 ) and sup 
k=n+ 1 o fc e{-l,°,l} 

k=l,...,n 




ic=i 


< K*n v 


for K[, K 2 > 0 and 7 , v > 0, then the convergence rate obtained in Theorem |3.3| is 


W^cx-^Wp = 0 (< 5 ( 1 °g( 1 /^)) T j as 0 

(cf. O Example 3.5]). 

Remark 3.6. Note that we always have weak convergence 

(3.11) Ae {k) —*• 0 in f as k —> 00 , 

because {e^ fc ^}fc g N converges weakly in £ 2 and A is weak-to-weak continuous since 
it is norm-to-norm continuous. In |9j Remark 2.5] it was shown that the slightly 
stronger condition 

(3.12) lim ||Ae (fe) || r = 0. 


enforces ||/^|| 


y* 


k—t 00 

00 in Theorems 


3.1 


and 3.3 


Obviously, condition (3.12) is satisfied if the underlying operator A is compact 
since compact operators map weakly convergent sequences to norm convergent ones 
(note that this property is equivalent to compactness of A if X is a Hilbert or at 
least a reflexive Banach space, cf. J25J Thm. 3.4.37]). On the other hand, one easily 
finds examples for noncompact operators which do not satisfy (3.12). Choose, e.g., 
X = Y = £ 2 and let A be the identity. Then ||Ae(0||v = 1 for all k € N. The 
question arises whether (3.12) is equivalent to compactness of A. The answer is 
‘no’ as the following example demonstrates. 
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Example 3.7 (Cesaro operator). Let X = Y = i 2 and define A : £ 2 —> 7 by 


(3.13) 


[^4x] n — ^ ^ %k • 

71 ' 


k=1 


This operator is injective and noncompact with nonclosed range (see on Solution 
177] or fSJ), but we have 


||Ae' 


(*0||2 - 


\\p 


= V — -A 0 if k 


OO. 


7=k 


Since with fA) ■= e W and / ^ := ke^ — (k — l)e^ k ~^ for k > 2 assumption (3.1) 
of Theorems |3.1| and |3.3| is satisfied, both convergence rates results apply to the 
specified operator. 

In the index function ipi in Theorem 3.1 the second sum is 

n n , 1 , 

E \\f w \\e = E . 

u —i u —i v " 

From 


k =1 


k=l 


J(k - l) 2 + k 2 > -4=(fc - 1 + k) = V2k - 4= 

~ V2 V2 


we even obtain a lower bound of the same order 


Eh / 

fc=i 


(fc)i 


p > 


\/2n(n + 1) n 


Vz 72 ' 


On the other hand we now show that the supremum in the definition of ip 2 can be 
estimated above by 


sup 

a»=e{-i,o,i} 

k=l,...,n 


Y, a *f [k) 


i 


e 2 


< 4n 3 / 2 . 
" 73 


At first we calculate 


E °fc/ (fe) = «i e( 1 ) +E k °fc e(fc) ~ E k 


^k-\- 1 ^ 


(k) 


k =1 


k =2 
n —1 


k =1 


= na n e^ n) + E k{a k - a k + i)e (/c) 
for arbitrary ai,..., a n . Thus, 


k =1 


E a */ (fc) 




\ 


n—1 


n2a n + ^ ~~ a ^+l) 2 ’ 


fc=l 


which attains its maximum over (ai,..., a n ) £ {—1, 0,1}" for a k = (—l) fe . Then 


sup 

a fc G{-l,0,l} 
k=l,...,n 

Here, Examples |3.4| and |3.5| apply and from these examples we see that the be¬ 
haviour of the estimated sum in ipi and of the supremum in directly carries over 
to the convergence rate. Thus, the slower growth of the supremum in comparison 


E«*/ (fe) 


k =1 


P 2 


\ 


■ 4 E k 2 = 


k=l 


-n 3 — n 2 + —n < 


73 


,3/2 
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to the faster growth of the sum yields a better rate for v = | based on Theorem 3.3 
than for v = 2 based on Theorem 13.11 


Example 3.8 (diagonal operator). For a comparison we briefly recall Example 2.6 
from [9], where A : X —> Y is a compact diagonal operator between the separable 
Hilbert spaces X and Y with the singular system {ok, u^ k \ v^}ken and Au^ = 
OkV^ k \ k € N. Then the decay rate of the singular value s Ok —> 0 for k —> oo 

k~ ( , C > 0 , 


k f 


characterizes the degree of ill-posedness of the equation (|l.l|). For Ok 
we have ||>' = ||||>- = Ok • 

with /<» = X v (k) and ||/( fc )|| y ~ 
some constant C > 0 


The link condition (3.1) is satisfied 
oo as k —> oo. Moreover, we have with 


sup 

a fc e{—1,0,1} 
k=l,...,n 




k =1 


Y 


\ 


n 1 n 1 

E4<E^-<^ 


,C+1 


cr, 

fc=l k 


fe=l 


dfc 


hence z/ = C + l>lin Examples |3.4| and |3.5| based on Theorem |3.3[ Note that the 


values C > | and consequently v > | correspond with the case of Hilbert-Schmidt 


operators H and v = | occurring in Example 3.7 is just a borderline case with 
respect to that fact. 


4. Ill-posedness of type I and II 

As suggested by M. Z. Nashed in [5B] we distinguish two types of ill-posedness 
for linear operator equations in a Banach space setting. Again, our focus is on 
injective operators. 

Definition 4.1. Let B : Z\ -A Z 2 be an injective and bounded linear operator 
mapping between the infinite dimensional Banach spaces Z\ and Z 2 ■ Then the 
operator equation 

(4.1) Bx = y 

is called well-posed if the range Tl{B) is a closed subset of Z 2 , consequently ill-posed 

- Z 2 

if the range is not closed, i.e. IZiB) X(B) . 

In the ill-posed case, the equation f f.l\ ) is called ill-posed of type I if the range 
7Z(B) contains an infinite dimensional closed subspace, and it is called ill-posed of 
type II otherwise. 


The two types of ill-posedness differ in the behavior of corresponding regularizers 
(cf. [55]) and with respect to smoothing properties of the linear operators B. If 
B := B\ : Z\ —> Z 2 is such that equation (4.1) proves to be ill-posed of type I 
and B := B 2 : Z\ —> Z 2 is such that equation (4.1) proves to be ill-posed of 
type II, then B 2 tends to be ‘more smoothing’ than B\. Namely, a range inclusion 
7 Z(B 2 ) C 1Z(Bi) may occur, but IZ(Bi) C TZ(B 2 ) cannot apply. We refer to 0 for 
consequences of range inclusions and in particular to Example 10.2 ibidem for the 
interplay of operators which characterize different types of ill-posedness. 

The following proposition shows that at least for operators B between Hilbert 
spaces Z\ and Z 2 the type of ill-posedness is determined by compactness properties 
of B. 
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Proposition 4.2. If the operator equation ( 4-1 ) is well-posed or ill-posed of type I, 
then the operator B is non-compact. Consequently, compactness of B implies ill- 
posedness of type II. 

If Z\ and Z 2 in equation (4-1) are Hilbert spaces and the equation is ill-posed, 


then the equation is ill-posed of type II if and only if B is compact. 


Proof. If the operator equation (4.1) is well-posed or ill-posed of type I, there is 


an infinite dimensional Banach space Z 2 included in the subspace P(B) of Z 2 with 
the same norm as in Z 2 - The preimage Z\ := B~ 1 (Z 2 ) is a Banach space included 
in Z\ with the same norm as in Z\. For a compact operator B , also its restriction 
B\^ 1 : Z\ —> Z 2 would be compact and moreover surjective. This contradicts the 
fact that a compact operator has only a closed range if it has a finite dimensional 
range, being a consequence of the non-compactness of the unit ball in a infinite 
dimensional Banach space. 


For Hilbert spaces Z\ and Z 2 and ill-posed equations (4.1), the equivalence of ill- 
posedness of type I and the non-compactness of B is well-known (cf. [20 Thm. 4.6] 
and El Lemma 5.8 and Theorem 5.9]). □ 


If compactness of B is replaced by strict singularity, the characterization of ill- 
posedness types can be made more precise for injective operators B. 

Definition 4.3. A bounded linear operator B between Banach spaces Z\ and Z 2 
is strictly singular if its restriction to an infinite dimensional subspace is never an 
isomorphism. 


Proposition 4.4. 

spaces Z\ and Z 2 - 
strictly singular. 


Let B be an injective bounded linear operator between Banach 
Then equation (4-1) is ill-posed of type II if and only if B is 


Proof. We show that there exists an isomorphic restriction of B to an infinite 
dimensional subspace of Z\ if and only if IZ(B) contains a closed infinite dimensional 
subspace. 

Obviously, if Z% is a closed infinite dimensional subspace of 1Z(B), then the 
corresponding preimage is of infinite dimension and the restriction of B to this 
preimage is an isomorphism. On the other hand, if there is an isomorphic restriction 
to an infinite dimensional subspace of Z \, then its image is also of infinite dimension 
and closed. □ 


Now we are going to apply Definition 4.1 to the equations (1.1) and (1.3) and 


to interpret the different cases. First we distinguish in the subsequent remark the 


possible cases arising in the context of equation (1.1). 


Remark 4.5. 


(a) Well-posed case: The equation (1.1) can be well-posed, which takes place 


if A is normally solvable. Linear Volterra integral equations of the second 
kind as well as more generally linear Fredholm integral equations of the 
second kind with appropriate kernels represent typical examples of this 
case, where X = Y = L 2 (Q) with some bounded and sufficiently regular 
domain H in K 1 , Z = 1,2,..., and the operator A is of the form A = I — K 
with the identity operator / and a compact operator K such that zero 
does not belong to the spectrum of the operator A. Normal solvability 
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also occurs if X = Y and A = I. Then solving (1.1), for given noisy data 
y s € Y, is the simplest case of a denoising problem (cf. Section^. 

(b) Ill-posed case: The equation (1.1) is ill-posed if the operator A fails to be 
normally solvable. This is just the case if the inverse A -1 : 71(A) CF->I 
is unbounded. 

(bl) Type I: Operators A for equations (1.1) which prove to be ill-posed 


(b2) 


(b3) 


of type I are non-compact and even not strictly singular. If Y is also a 
Hilbert space, all ill-posed equations 0 with non-compact operator 
A are of this type. Multiplication operators in X = Y = L 2 (a,b) with 
L°°(a , b)-multiplier functions possessing essential zeros and linear con¬ 
volution operators in X = Y = L 2 (M. 1 ) with square-integrable kernels 
represent typical examples for this case. Furthermore, the Hausdorff 
moment problem with A : L 2 (0,1) —> £ 2 (cf. [|18j Example 3.2]) is of 
this type. 

Type II, non-compact: If Y is not a Hilbert space then there ex¬ 
ist strictly singular operators A with nonclosed range which are not 
compact, for example the embedding operators from f 2 to £ q with 
2 < q <o o (cf. US Theorem (a)]). This also leads to ill-posed equa¬ 
tions (1.1) of type II. Such operators A can have a non-separable range 


71(A) (cf. [Hi Remark on p. 335]). 

Type II, compact: The equation 0. is ill-posed of type II if A : 
X —► Y is a compact operator. Then A is strictly singular and the 
range 71(A) is a separable space. Typical examples with compact 
operators A are linear Fredholm and Volterra integral equations of 
the first kind with square-integrable kernels in L 2 -spaces X and Y 
over bounded and sufficiently regular domains in M. 1 . Moreover, all 
bounded linear operators A : f 2 -A £ q are compact for 1 < q < 2 as 
was mentioned in m- In particular, if also Y is a Hilbert space, then 
vice versa ill-posedness of type II requires compactness of A. 


The diagram in Figure 1 illustrates the different cases in Remark |4.5| and the re¬ 
sults of this section concerning the relations between compactness, strict singularity 
and type of ill-posedness for injective forward operators. By the way, we should 
mention that there exist non-injective strictly singular operators possessing a range 
which contains an infinite dimensional closed subspace (cf. 1141 first example]). 

A scenario completely different from Remark H75] occurs for equation (1.3) due to 


the composition structure (1.4) of the operator A. The fact that the non-compact 


embedding operator £2 is strictly singular (cf. [21 Theorem]) prevents the occur¬ 
rence of well-posedness and ill-posedness of type I in the context of this equation. 


Proposition 4.6. Under the assumptions stated above equation (1.3) is 
always ill-posed of type II. 

Proof. Taking into account Proposition |4.4| we only have to show that A is always 


strictly singular. But this follows immediately from the composition structure (1.4) 


and the two facts that £2 is strictly singular (see [2]) and that the composition of a 
strictly singular operator with a bounded linear operator is again strictly singular. 

□ 
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Figure 1. Relations between strict singularity, compactness and 
type of ill-posedness for equations (4.1) with injective bounded 
linear operator. 


5. The special case of denoising 


Finally, we apply our results to a typical denoising problem. Given a noisy signal 
one wants to remove the noise. For this purpose one decomposes the signal with 
respect to a wavelet basis (or any other orthonormal system) and tries to find a 
sparse approximation with respect to this basis. Thus, in our setting we choose A to 
be the identity on £ 2 . Since in some applications it might be reasonable to measure 
the noise in a weaker norm we extend Y to £ q with 2 < q < oo. Then A := £ q is 
the embedding of f 1 into £ q . In the sequel we only look at A and therefore extend 
the feasible values for q to 1 < q < oo. 


The minimization problem (1.5) now reads as 


(5.1) 


1 

P' 


\\£q x — y s \\ei + a ||x||^i —> min, subject to iG 1 , 


where the exact signal x' is assumed to be nearly sparse, i.e. x f £ i 1 . From the 
computational point of view it seems to be helpful to apply p := q for the exponent 
in the misfit term of (5.1) whenever 1 < q < oo. If we measure the error after 
denosing in the ^-norm as \\x s a — a^||p, then the chances of having small errors 
improve with decreasing values q , since the strength of the norm in Y grows if q 
decreases. 

Proposition 5.1. For the embedding operator A = £ q from i 1 to £ q with 1 < q < oo 
equation (1.3) is ill-posed of type II. We have weak convergence Ae^ —*• 0 ifk —> oo 
for 1 < q < oo , but no convergence in norm. For all 1 < q < oo and all k £ N the 
link condition (3.1) is satisfied with /( fc ' = e^. Thus, Theorems 3.1 and 3.3 apply 
and the corresponding index functions are 


(5.2) 

and 


Ti 


(t) = 2 Sl £ |a; ^ l + tn 


\k=n -\-1 


if 1 < q < oo 


Mt) = 2 inf ( \ x l\+ tn<> 


\k=n -\-1 


(5.3) 
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with 


1 — 1, if 1 < q < oo, 




1 , 


i/ g = OO. 


Proof. The ill-posedness of type II is a consequence of Proposition 4.6 whenever 
2 < q < oo, because the injective and bounded embedding operator from £ 2 to 
£ q plays here the role of A. On the other hand, the non-existence of an infinite 
dimensional closed subspace in £ q for 1 < q < 2 included in £ l also follows from the 
theorem in m, since the corresponding embedding operators are strictly singular. 

It is evident that does not converge to zero in the norm of £ q , but we have 
e ( fe ) 0 in £ q for all 1 < q < oo. Also the validity of (3.1) is evident. 


The index functions <p± and ip2 in the convergence rates theorems can be com¬ 
puted easily for the special case under consideration. □ 

Example 5.2 (Holder rates). If the decay rate x\. —> 0 as k — > oo of the remaining 
solution coefficients is of power type 

OO 

(5.4) |xj[.| < C x t k G N, or equivalently ^ \x\\ < K x t n _/i , n G N, 

k=n +1 

with constants fi > 0 and C x t, K x t > 0, we immediately derive from Proposi¬ 


tion 5.1 and Example 3.4 the Holder convergence rates for the denoising problem 
with forward operator A = £ q as 


(5.5) 

and 

(5.6) 


-x^\\ e = O ( 


\x S a - = O (Sk+i'J 


as S v 0 


as 5 —> 0 


if 1 < q < oo 


if 


q = Too. 


As expected the rate grows if q decreases, i.e., if the noise is measured in a stronger 
norm. On the other hand, the borderline case 9 = 1 leads to a well-posed equation 


(1.3). In this case the index function ip 2 attains the form 


w|,|=! 3 £ 

\k=n+l 


x' k \+t\ =2 1 


and the corresponding rate is 

ll^’ct — x^Wz 1 = O (<5) as <5 —>• 0 
which is typical for well-posed situations. 


if 


if 9 = 1 


9 = 1, 


The example also shows that the improved index function if 2 from Theorem 3.3 
indeed provides a better convergence rate for 1 < q < oo than the original index 
function tpi from Theorem |3.1| Note that ipi in Proposition [54] is for all q the same 
function as </?2 with q = oo. 

At the end we should mention that the rate results ( |5.5[ ) and ( |5.6[ ) yield the values 
0 < v < 1 in formula (3.10) from Example 3.4 Consequently, the Holder rates in 
Examples |3.7| and 


| with v > 1 are always lower than the observed rates for 
the denoising case, which indicates a lower degree of ill-posedness for the denoising 
problem. 
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